Skip to content

[Bug] ESQL Remote Validation Ignoring Rule Min-Stack#6223

Open
terrancedejesus wants to merge 3 commits into
mainfrom
6213-bug-esql-remote-validation-for-unknown-indices
Open

[Bug] ESQL Remote Validation Ignoring Rule Min-Stack#6223
terrancedejesus wants to merge 3 commits into
mainfrom
6213-bug-esql-remote-validation-for-unknown-indices

Conversation

@terrancedejesus
Copy link
Copy Markdown
Contributor

@terrancedejesus terrancedejesus commented Jun 1, 2026 •
edited
Loading

Pull Request

Issue link(s):

Summary - What I changed

Adds a small change to ignore stack version validation in ESQLValidator.remote_validate_rule() if the version is < the defined min_stack_version of the rule. This causes ESQL remote validation errors where the index pattern is not found in locally (integration manifests/schema) data, ultimately failing. Please see related issue for more details.

How To Test

  1. Checkout new-rule/azure-ad-graph-potential-roadrecon-enum
  2. Run python -m detection_rules view-rule rules/integrations/azure/discovery_aad_graph_roadrecon_aitohttp_enumeration.toml --esql-remote-validation
  3. Observe error
Error (EsqlUnknownIndexError): Unknown index pattern(s): logs-azure.aadgraphactivitylogs-*. Known patterns: logs-azure.eventhub*, logs-azure.platformlogs*, logs-azure.activitylogs-*, logs-azure.platformlogs-* ...
  1. Add debugger config
  {
      "name": "esql remote validation repro",
      "type": "debugpy",
      "request": "launch",
      "module": "detection_rules",
      "console": "integratedTerminal",
      "justMyCode": false,
      "cwd": "${workspaceFolder}",
      "args": [
          "view-rule",
          "rules/integrations/azure/discovery_aad_graph_roadrecon_aiohttp_enumeration.toml",
          "--esql-remote-validation"
      ]
  }
  1. Set breakpoint at rule_validators.py#L928 (notice it starts at lowest 8.19.0 and increments up) -> note this the issue and should not be evaluated here
  2. Continue --> error
  3. Add the suggested fix to this branch and run again, notice only 9.3+ is evaluated in the for version in versions: loop

Checklist

  • Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
  • Added the meta:rapid-merge label if planning to merge within 24 hours
  • Secret and sensitive material has been managed correctly
  • Automated testing was updated or added to match the most common scenarios
  • Documentation and comments were added for features that require explanation

Contributor checklist

@terrancedejesus terrancedejesus linked an issue Jun 1, 2026 that may be closed by this pull request
[Bug] ESQL Remote Validation for Unknown Indices #6213
Open
@terrancedejesus terrancedejesus self-assigned this Jun 1, 2026
@terrancedejesus terrancedejesus added bug Something isn't working python Internal python for the repository labels Jun 1, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

Bug - Guidelines

These guidelines serve as a reminder set of considerations when addressing a bug in the code.

Documentation and Context

  • Provide detailed documentation (description, screenshots, reproducing the bug, etc.) of the bug if not already documented in an issue.
  • Include additional context or details about the problem.
  • Ensure the fix includes necessary updates to the release documentation and versioning.

Code Standards and Practices

  • Code follows established design patterns within the repo and avoids duplication.
  • Ensure that the code is modular and reusable where applicable.

Testing

  • New unit tests have been added to cover the bug fix or edge cases.
  • Existing unit tests have been updated to reflect the changes.
  • Provide evidence of testing and detecting the bug fix (e.g., test logs, screenshots).
  • Validate that any rules affected by the bug are correctly updated.
  • Ensure that performance is not negatively impacted by the changes.
  • Verify that any release artifacts are properly generated and tested.
  • Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

  • Verify that the bug fix works across all relevant environments (e.g., different OS versions).
  • Confirm that the proper version label is applied to the PR patch, minor, major.

eric-forte-elastic
eric-forte-elastic reviewed Jun 2, 2026
View reviewed changes
Comment thread detection_rules/rule_validators.py
# stream) and raises spurious EsqlUnknownIndexError. Fall back to the minimum supported
# stack when the rule does not pin a min_stack_version.
min_stack = Version.parse(
str(metadata.min_stack_version or get_min_supported_stack_version()),
Copy link
Copy Markdown
Contributor

@eric-forte-elastic eric-forte-elastic Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit. Is get_min_supported_stack_version needed? Its loading from the schema and get_stack_versions also loads verbatim from the schema so it should never be different right?

Copy link
Copy Markdown
Contributor

@eric-forte-elastic eric-forte-elastic Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also get_min_supported_stack_version already returns a Version object so the cast to string and then back to Version is unnecessary.

eric-forte-elastic
eric-forte-elastic reviewed Jun 2, 2026
View reviewed changes
Comment thread detection_rules/rule_validators.py
for version in get_stack_versions():
if version in mappings_lookup:
continue
if Version.parse(version) < min_stack:
Copy link
Copy Markdown
Contributor

@eric-forte-elastic eric-forte-elastic Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If for instance we min stack a rule to 9.4.1, it may not go through this section of the validation if the latest stack version is 9.4.0. Not inherently a problem, just that we need to be sure that the min stacks will not be a min stacked to a version newer than the latest version in the manifest.

eric-forte-elastic
eric-forte-elastic approved these changes Jun 2, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@eric-forte-elastic eric-forte-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual review, confirmed functionality accomplishes goal. Few comments, but no blockers.

Image

@terrancedejesus terrancedejesus marked this pull request as ready for review June 2, 2026 14:54
@Mikaayenson
Copy link
Copy Markdown
Contributor

Mikaayenson commented Jun 2, 2026

@terrancedejesus After looking at the issues / comments/ etc, it appears there are really two themes.

  1. The central issue is that we don't support patch validation on minors like @eric-forte-elastic mentioned, where: stack-schema-map.yaml + get_stack_versions() + prepare_integration_mappings() using Version.parse(stack_version) literally will get us something like 8.19.0 based on 8.19. Even if we could ever min-stack a rule to 8.19.10, it would still validate at 8.19.0, and you'll still get the error. Eric's proposed direction to represent latest patch per release line in stack versions is the structurally correct fix for this bug.
  2. Adjusting our remote ESQL validation (what's described in this PR). IINM if we properly address the first issue and properly min stack the rules, we then don't need this second change. Whereas if we just do this quick stop gap change, it doesn't fix all the issues and introduces tech debt (and I think you'll have to bump minstacks on your AAD rules).

Can you look deeper into these to decide if we should still move forward with this change? And if it will work for all of your AAD rules?

@eric-forte-elastic
Copy link
Copy Markdown
Contributor

eric-forte-elastic commented Jun 3, 2026

@terrancedejesus After looking at the issues / comments/ etc, it appears there are really two themes.

  1. The central issue is that we don't support patch validation on minors like @eric-forte-elastic mentioned, where: stack-schema-map.yaml + get_stack_versions() + prepare_integration_mappings() using Version.parse(stack_version) literally will get us something like 8.19.0 based on 8.19. Even if we could ever min-stack a rule to 8.19.10, it would still validate at 8.19.0, and you'll still get the error. Eric's proposed direction to represent latest patch per release line in stack versions is the structurally correct fix for this bug.
  2. Adjusting our remote ESQL validation (what's described in this PR). IINM if we properly address the first issue and properly min stack the rules, we then don't need this second change. Whereas if we just do this quick stop gap change, it doesn't fix all the issues and introduces tech debt (and I think you'll have to bump minstacks on your AAD rules).

Can you look deeper into these to decide if we should still move forward with this change? And if it will work for all of your AAD rules?

Quick note to this, part of the underlying problem is that our stack schema map file is by hand kept only up to date for major.minor and patch is always 0, even when this is not correct. E.g. out stack schema map says 8.19.0, but we never release to 8.19.0 its 8.19.latest. So if we even just kept this mapping up to date, this would be a much simpler issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eric-forte-elastic eric-forte-elastic eric-forte-elastic approved these changes

@Mikaayenson Mikaayenson Awaiting requested review from Mikaayenson Mikaayenson is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

@terrancedejesus terrancedejesus

Labels

backport: auto bug Something isn't working patch python Internal python for the repository

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] ESQL Remote Validation for Unknown Indices

3 participants

@terrancedejesus @Mikaayenson @eric-forte-elastic